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Moment Generating Functions 


6.1.3 Moment Generating Functions 

Here, we will introduce and discuss moment generating functions (MGFs). Moment 
generating functions are useful for several reasons, one of which is their application to 
analysis of sums of random variables. Before discussing MGFs, let's define moments. 


Definition 6.2. The nth moment of a random variable X is defined to be E[X n ] . The nth 
central moment of X is defined to be E [( X — EX) 71 ]. 


For example, the first moment is the expected value -E7[JT]. The second central moment is the 
variance of X . Similar to mean and variance, other moments give useful information about 
random variables. 

The moment generating function (MGF) of a random variable X is a function Mx( s ) 
defined as 


M x (s) = E 


jX 


We say that MGF of X exists, if there exists a positive constant a such that Mx(s) is finite 
for all S G [—a, a]. 


Before going any further, let's look at an example. 


Example 6.3 

For each of the following random variables, find the MGF. 
a. X is a discrete random variable, with PMF 

1 

2 



b Y is a Uniform(0, 1 ) random variable. 
• Solution 

o a. For X, we have 


https://www.probabilitycourse.com/chapter6/6_1_3_moment_functions.php 


1/8 



9/17/2018 


Moment Generating Functions 


Mx{s) = E 




1 2 9 

—e s + -e 2s . 
3 3 


which is well-defined for all S E M. 
b. For Y, we can write 

M y {s) = E 


sY 


f 


e sy dy 
r o 

e s - 1 


s 

Note that we always have My{ 0 ) = E[e®' Y ] — 1 , thus My(s ) is 
also well-defined for all s G M. 


Why is the MGF useful? There are basically two reasons for this. First, the MGF of X gives 
us all moments of X. That is why it is called the moment generating function. Second, the 
MGF (if it exists) uniquely determines the distribution. That is, if two random variables have 
the same MGF, then they must have the same distribution. Thus, if you find the MGF of a 
random variable, you have indeed determined its distribution. We will see that this method is 
very useful when we work on sums of several independent random variables. Let's discuss 
these in detail. 

Finding Moments from MGF: 

Remember the Taylor series for e x : for 

e* = 1 + x + 


Now, we can write 

oo 

e‘ x = E 

fe=0 


all X el, we have 


” x k 


rf »" /yj 'J _ 

~2!" + 3T + "“ 


k =0 


k\ 


(. sX) k _ g X k s k 


k\ 


k =0 


k\ 


Thus, we have 
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oo k 

M x (s)=E[e‘ x \ = Y J E[X k \ S 1 r- 

k=0 K ‘ 

k 

We conclude that the kth moment of X is the coefficient of in the Taylor series of 
Mx{ s )- Thus, if we have the Taylor series of Mx{s), we can obtain all moments of X. 


Example 6.4 

ifE ~ Uniform(0, 1), find E[Y k ] using My (s). 
• Solution 

o We found My(s) in Example 6.3, so we have 


M y {s) 


e s - 1 


s 



-1 



s 


k 


k\ 


OO 


E 

k =1 


oo 


E 

k =0 


s k-l 

“IT 

1 s k 
k + 1 k\ 


Thus, the coefficient of in the Taylor series for My{s) is so 


k+1 


E[X k ] = 


1 

k + 1 


We remember from calculus that the coefficient of in the Taylor series of Mx(s) is 

obtained by taking the kth derivative of Mx{s ) and evaluating it at S — 0 . Thus, we can 
write 

E[ X = £ m W)I»=o- 
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We can obtain all moments of X^ from its MGF: 

°o k 

M x (s) = ^E[X k ]-, 

k =0 

E[xk \ = 


Example 6.5 

Let X ~ Exponential A). Find the MGF of X , Mx(s), and all of its moments, 

E[X k \. 

• Solution 

o Recall that the PDF of X is 


fx{x) = Ae Xx u(x ), 
where U\x) is the unit step function. We conclude 

M x (s) = E[e sX ] 

A e-^e ax dx 


L 


A 


-l CO 


A — s 


3 —(A —s)x 


, for s < A 


J o 


A 


A — s 


, for s < A. 


Therefore, iVfx( s ) exists for all S < A. To find the moments of X, we can 
write 
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Mx(s) 


X 

A — s 

1 



OO 

E 

k= 0 


k\ s k 

A k k\ 



< 1 


We conclude that 


E[X k ] 


k[_ 

A*’ 


for /c = 0,1,2,... 


Example 6.6 

Let X ~ Poisson(X). Find the MGF of X , Mx(s). 


• Solution 

o We have 


Thus, 


A 


Px{k) = e x — , for fe = 0,1,2,... 


Mx{s) = E[e sX \ 


OO 


e sk e~ x 


X k 


“ fc! 

-- A E 


o° \ k 
sk A 


fc! 

(Ae s ) / 

"fcT 


k=0 
00 f\„s\k 


-- A E 

= e _A e AeS (Taylor series for e*) 
= e^-^, for alls e R. 
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As we discussed previously, the MGF uniquely detennines the distribution. This is a very 
useful fact. We will see examples of how we use it shortly. Right now let's state this fact more 
precisely as a theorem. We omit the proof here. 

Theorem 6.1 Consider two random variables X and Y . Suppose that there exists a positive 
constant c such that MGFs of X and Y are finite and identical for all values of s in [— c, c]. 
Then, 

Fx{t ) = Fy(t ), for all t E R. 

Example 6.7 

For a random variable X, we know that 

Mx(s) = fors E (-2,2). 

2i s 

Find the distribution of X . 

• Solution 

o We note that the above MGF is the MGF of an exponential random variable with 
A = 2 (Example 6.5). Thus, we conclude that X ~ Exponential (2). 


Sum of Independent Random Variables: 

Suppose X\, X‘2, ..., X n are n independent random variables, and the random variable Y is 
defined as 

Y = X x + X 2 + • • • + X n . 

Then, 

M y (s) = E[e sY ] 

- 1+^2+- ■ -+X ra ) j 

= E[e sXl e sXl • • • e sXn ] 

— E[e sXl }E[e sX2 ] • • • E[e sXn ] (since the X^s are independent) 
= M Xl (s)M X 2 (s)---M Xn (s). 

If X\, X<2 ,.... X n are n independent random variables, then 

M Xl +x 2 +---+x n (s ) = M Xl (s)Mx 2 (s) ■ • • M Xn {s ). 
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Example 6.8 

if X ~ Binomial{n , p ) find the MGF of X. 

• Solution 

o We can solve this question directly using the definition of MGF, but an easier way 
to solve it is to use the fact that a binomial random variable can be considered as 
the sum of TL independent and identically distributed (i.i.d.) Bernoulli random 
variables. Thus, we can write 

X — X\ + X 2 + • • • + X n , 
where X\ ~ Bernoulli {p). Thus, 

Mx(s) = M Xl ( s)Mx 2 (s) • • • M Xn (s) 

= [MxM) (since the X^s are i.i.d.) 


Also, 


Mx 1 (s) = E[e sXl ] — pe s + 1 — p. 


Thus, we conclude 


Mx(s) = (; pe s + l-p) n . 


Example 6.9 

Using MGFs prove that if X ~ Binomial(m , p) and Y ~ Binomial(n , p) are 
independent, then X + Y ~ Binomial(m + n,p). 

• Solution 

o We have 


M x (s) = (; pe s + 1 -p) m , 
M y {s) = (pe s + 1 -p) n ■ 

Since X and Y are independent, we conclude that 

M x +y{s) = Mx(s)M y (s) 

= (pe’ + l-p) m+ \ 
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which is the MGF of a Binomial (m + 71, p) random variable. Thus, 
X + Y ~ Binomial(m + n,p). 


previous 
next —> 
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